A Nationwide Network of Health AI Assurance Laboratories.

Importance: Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed. Observations: While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings. Conclusion and Relevance: The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

Assessing a New Prescreening Score for the Simplified Evaluation of the Clinical Quality and Relevance of eHealth Apps: Instrument Validation Study.

BACKGROUND: In 2020, more than 250 eHealth solutions were added to app stores each day, or 90,000 in the year; however, the vast majority of these solutions have not undergone clinical validation, their quality is unknown, and the user does not know if they are effective and safe. We sought to develop a simple prescreening scoring method that would assess the quality and clinical relevance of each app. We designed this tool with 3 health care stakeholder groups in mind: eHealth solution designers seeking to evaluate a potential competitor or their own tool, investors considering a fundraising candidate, and a hospital clinician or IT department wishing to evaluate a current or potential eHealth solution. OBJECTIVE: We built and tested a novel prescreening scoring tool (the Medical Digital Solution scoring tool). The tool, which consists of 26 questions that enable the quick assessment and comparison of the clinical relevance and quality of eHealth apps, was tested on 68 eHealth solutions. METHODS: The Medical Digital Solution scoring tool is based on the 2021 evaluation criteria of the French National Health Authority, the 2022 European Society of Medical Oncology recommendations, and other provided scores. We built the scoring tool with patient association and eHealth experts and submitted it to eHealth app creators, who evaluated their apps via the web-based form in January 2022. After completing the evaluation criteria, their apps obtained an overall score and 4 categories of subscores. These criteria evaluated the type of solution and domain, the solution's targeted population size, the level of clinical assessment, and information about the provider. RESULTS: In total, 68 eHealth solutions were evaluated with the scoring tool. Oncology apps (22%, 20/90) and general health solutions (23%, 21/90) were the most represented. Of the 68 apps, 32 (47%) were involved in remote monitoring by health professionals. Regarding clinical outcomes, 5% (9/169) of the apps assessed overall survival. Randomized studies had been conducted for 21% (23/110) of the apps to assess their benefit. Of the 68 providers, 38 (56%) declared the objective of obtaining reimbursement, and 7 (18%) out of the 38 solutions seeking reimbursement were assessed as having a high probability of reimbursement. The median global score was 11.2 (range 4.7-17.4) out of 20 and the distribution of the scores followed a normal distribution pattern (Shapiro-Wilk test: P=.33). CONCLUSIONS: This multidomain prescreening scoring tool is simple, fast, and can be deployed on a large scale to initiate an assessment of the clinical relevance and quality of a clinical eHealth app. This simple tool can help a decision-maker determine which aspects of the app require further analysis and improvement.

External quality assessment of COVID-19 real time reverse transcription PCR laboratories in India.

Sudden emergence and rapid spread of COVID-19 created an inevitable need for expansion of the COVID-19 laboratory testing network across the world. The strategy to test-track-treat was advocated for quick detection and containment of the disease. Being the second most populous country in the world, India was challenged to make COVID-19 testing available and accessible in all parts of the country. The molecular laboratory testing network was augmented expeditiously, and number of laboratories was increased from one in January 2020 to 2951 till mid-September, 2021. This rapid expansion warranted the need to have inbuilt systems of quality control/ quality assurance. In addition to the ongoing inter-laboratory quality control (ILQC), India implemented an External Quality Assurance Program (EQAP) with assistance from World Health Organization (WHO) and Royal College of Pathologists, Australasia. Out of the 953 open system rRTPCR laboratories in both public and private sector who participated in the first round of EQAP, 891(93.4%) laboratories obtained a passing score of > = 80%. The satisfactory performance of Indian COVID-19 testing laboratories has boosted the confidence of the public and policy makers in the quality of testing. ILQC and EQAP need to continue to ensure adherence of the testing laboratories to the desired quality standards.

Methodologic and Reporting Quality of Economic Evaluations in Hand and Wrist Surgery: A Systematic Review.

BACKGROUND: Economic evaluations can inform decision-making; however, previous publications have identified poor quality of economic evaluations in surgical specialties. METHODS: Study periods were from January 1, 2006, to April 20, 2020 (methodologic quality) and January 1, 2014, to April 20, 2020 (reporting quality). Primary outcomes were methodologic quality [Guidelines for Authors and Peer Reviewers of Economic Submissions to The BMJ (Drummond's checklist), 33 points; Quality of Health Economic Studies (QHES), 100 points; Consensus on Health Economic Criteria (CHEC), 19 points] and reporting quality (Consolidated Health Economic Evaluation Standards (CHEERS) statement, 24 points). RESULTS: Forty-seven hand economic evaluations were included. Partial economic analyses (i.e., cost analysis) were the most common (n = 34; 72 percent). Average scores of full economic evaluations (i.e., cost-utility analysis and cost-effectiveness analysis) were: Drummond's checklist, 27.08 of 33 (82.05 percent); QHES, 79.76 of 100 (79.76 percent); CHEC, 15.54 of 19 (81.78 percent); and CHEERS, 20.25 of 24 (84.38 percent). Cost utility analyses had the highest methodologic and reporting quality scores: Drummond's checklist, 28.89 of 35 (82.54 percent); QHES, 86.56 of 100 (86.56 percent); CHEC, 16.78 of 19 (88.30 percent); and CHEERS, 20.8 of 24 (86.67 percent). The association (multiple R) between CHEC and CHEERS was strongest: CHEC, 0.953; Drummond's checklist, 0.907; and QHES, 0.909. CONCLUSIONS: Partial economic evaluations in hand surgery are prevalent but not very useful. The Consensus on Health Economic Criteria and Consolidated Health Economic Evaluation Standards should be used in tandem when undertaking and evaluating economic evaluation in hand surgery.

The sixth edition of the WHO Laboratory Manual for the Examination and Processing of Human Semen: ensuring quality and standardization in basic examination of human ejaculates.

A basic semen investigation has established principles that are necessary for ascertaining reliable and internationally comparable results. Although these principles have been present in the WHO manual since its inception, the baseline issue across most published studies and practice in reproductive medicine (in which the male is considered) is repetitive failure to adhere to these principles, thereby leading to relevant comparable data and accuracy. To address this failure, the sixth edition of the WHO manual includes revised basic methods, and a complementary formal standard of the International Standards Organization (ISO23162:2021) for basic semen examination has been published. Perhaps the most significant change in the sixth edition is the reintroduction of the four-category distinction of sperm motility, which causes additional work for laboratories in changing reporting parameters but is clinically important. Another essential change is the widened focus from mainly a prognostic tool for medically assisted reproduction to additionally raising awareness of semen examination as a measure of male reproductive functions and general male health.

The age-specific differences in histopathological tumor characteristics and TNM classification of breast carcinomas in Quality assured mamma diagnostic (QuaMaDi) program in the state of Schleswig-Holstein in Germany.

BACKGROUND: We explored the hypothesis that high-quality standards in diagnostic mammography can lead to an early diagnosis of breast cancers and identifies at risk populations outside screening programs. The histopathological features and distribution of the TNM classification were examined in relation to patient age in a large group of women with breast cancers participating in the Quality Assured Mamma Diagnostic (QuaMaDi) program of the state of Schleswig-Holstein. PATIENTS AND METHODS: Surgical pathological reports were studied for clinicopathological characteristics, receptor status, molecular subtype and tumor stage. The analysis was conducted by dividing the study population into three age groups: women under 50 years (pre-screening), 50-69 years (peri-screening) and over 70 years (post-screening). RESULTS: 7.111 biopsies and 2.887 resection specimens were included. Breast cancer was diagnosed in 4.241 (59.7%) cases, one fourth of them in women < 50 years. Elderly women (> 70 years) had more well-differentiated, estrogen receptor (ER)-positive and HER2-negative carcinomas, whereas younger women (< 50 years) tended to have more poorly differentiated, ER negative, and HER2-positive carcinomas. 47% of breast carcinoma were luminal B tumors and were most common regardless of age. 70.4% of resected specimen had pT1 stage. Nodal negative were 71.2%. CONCLUSION: In QuaMaDi breast cancer was diagnosed at an early and potentially curable stage of the disease due to high-quality standards in diagnostic mammography. In addition, regardless of age, an increased number of prognostically unfavorable molecular subtypes were detected. Thus, QuaMaDi helps to identify at risk populations. QuaMaDi significantly improves diagnostic mammography and complements mammography screening programs.

Face-Validated Quality Indicators for Appropriateness of End-of-Life Care in Children with Serious Illness: A Study Using the RAND/University of California at Los Angeles Appropriateness Method.

OBJECTIVE: To develop and face-validate population-level indicators for potential appropriateness of end-of-life care, for children with cancer, neurologic conditions, and genetic/congenital conditions, to be applied to administrative health data containing medication and treatment variables. STUDY DESIGN: Modified RAND/University of California at Los Angeles appropriateness method. We identified potential indicators per illness group through systematic literature review, scoping review, and expert interviews. Three unique expert panels, a cancer (n = 19), neurology (n = 21), and genetic/congenital (n = 17) panel, participated in interviews and rated indicators in individual ratings, group discussions, and second individual ratings. Each indicator was rated on a scale from 1 to 9 for suitability. Consensus was calculated with the interpercentile range adjusted for symmetry formula. Indicators with consensus about unsuitability were removed, those with consensus about suitability were retained, and those with lack of consensus deliberated in the group discussion. Experts included pediatricians, nurses, psychologists, physiotherapists, pharmacologists, care coordinators, general practitioners, social workers from hospitals, care teams, and general practice. RESULTS: Literature review and expert interviews yielded 115 potential indicators for cancer, 111 for neurologic conditions, and 99 for genetic/congenital conditions. We combined similar indicators, resulting in respectively 36, 32, and 33 indicators per group. Expert scoring approved 21 indicators for cancer, 24 for neurologic conditions, and 23 for genetic/congenital conditions. CONCLUSIONS: Our indicators can be applied to administrative data to evaluate appropriateness of children's end-of-life care. Differences from adults' indicators stress the specificity of children's end-of-life care. Individual care and remaining aspects, such as family support, can be evaluated with complementary tools.

How to assure the quality of clinical records? A 7-year experience in a large academic hospital.

INTRODUCTION: Clinical record (CR) is the primary tool used by healthcare workers (HCWs) to record clinical information and its completeness can help achieve safer practices. CR is the most appropriate source in order to measure and evaluate the quality of care. In order to achieve a safety climate is fundamental to involve a responsive healthcare workforce thorough peer-review and feedbacks. This study aims to develop a peer-review tool for clinical records quality assurance, presenting the seven-year experience in the evolution of it; secondary aims are to describe the CR completeness and HCWs' diligence toward recording information in it. METHODS: To assess the completeness of CRs a peer-review tool was developed in a large Academic Hospital of Northern Italy. This tool included measurable items that examined different themes, moments and levels of the clinical process. Data were collected every three months between 2010 and 2016 by appointed and trained HCWs from 42 Units; the hospital Quality Unit was responsible for of processing and validating them. Variations in the proportion of CR completeness were assessed using Cochran-Armitage test for trends. RESULTS: A total of 9,408 CRs were evaluated. Overall CR completeness improved significantly from 79.6% in 2010 to 86.5% in 2016 (p<0.001). Doctors' attitude showed a trend similar to the overall completeness, while nurses improved more consistently (p<0.001). Most items exploring themes, moments and levels registered a significant improvement in the early years, then flattened in last years. Results of the validation process were always above the cut-off of 75%. CONCLUSIONS: This peer-review tool enabled the Quality Unit and hospital leadership to obtain a reliable picture of CRs completeness, while involving the HCWs in the quality evaluation. The completeness of CR showed an overall positive and significant trend during these seven years.

Comparison of sliding window and field-in-field techniques for tangential whole breast irradiation using the Halcyon and Synergy Agility systems.

BACKGROUND: To implement a tangential treatment technique for whole breast irradiation using the Varian Halcyon and to compare it with Elekta Synergy Agility plans. METHODS: For 20 patients two comparable treatment plans with respect to dose coverage and normal tissue sparing were generated. Tangential field-in-field treatment plans (Pinnacle/Synergy) were replanned using the sliding window technique (Eclipse/Halcyon). Plan specific QA was performed using the portal Dosimetry and the ArcCHECK phantom. Imaging and treatment dose were evaluated for treatment delivery on both systems using a modified CIRS Phantom. RESULTS: The mean number of monitor units for a fraction dose of 2.67 Gy was 515 MUs and 260 MUs for Halcyon and Synergy Agility plans, respectively. The homogeneity index and dose coverage were similar for both treatment units. The plan specific QA showed good agreement between measured and calculated plans. All Halcyon plans passed portal dosimetry QA (3%/2 mm) with 100% points passing and ArcCheck QA (3%/2 mm) with 99.5%. Measurement of the cumulated treatment and imaging dose with the CIRS phantom resulted in lower dose to the contralateral breast for the Halcyon plans. CONCLUSIONS: For the Varian Halcyon a plan quality similar to the Elekta Synergy device was achieved. For the Halcyon plans the dose contribution from the treatment fields to the contralateral breast was even lower due to less interleaf transmission of the Halcyon MLC and a lower contribution of scattered dose from the collimator system.

Health Economic Impact of a Multicenter Quality-of-Care Initiative for Reducing Unplanned Healthcare Utilization Among Patients With Inflammatory Bowel Disease.

INTRODUCTION: A multicenter adult inflammatory bowel disease learning health system (IBD Qorus) implemented clinical care process changes for reducing unplanned emergency department visits and hospitalizations using a Breakthrough Series Collaborative approach. METHODS: Using Markov decision models, we determined the health economic impact of participating in the Collaborative from the third-party payer perspective. RESULTS: Across all 23 sites, participation in the Collaborative was associated with lower annual costs by an average of $2,528 ± $233 per patient when compared with the baseline period. DISCUSSION: Implementing clinical care process changes using a Collaborative approach was associated with overall cost savings. Future work should examine which specific interventions are most effective and whether such cost savings are sustainable.

Analysis of clinical patient-specific pre-treatment quality assurance with the new helical tomotherapy platform, following the AAPM TG-218 report.

PURPOSE: This study presents patient-specific quality assurance (QA) results from the first 395 clinical cases for the new helical TomoTherapy® platform (Radixact) coupled with dedicated Precision TPS. METHODS: The passing rate of the Gamma Index (GP%) of 395 helical QA of patient-specific tomotherapy, acquired with ArcCHECK, is presented, analysed and correlated to various parameters of the plan. Following TG-218 recommendations, the clinic specific action limit (ALcs) and tolerance limit (TLcs) were calculated for our clinic and monitored during the analysed period. RESULTS: The mean values ​​(± 1 standard deviation) of GP% (3%/2 mm) (both global and local normalization) are: 97.6% and 90.9%, respectively. The proposed ALcs and TLcs, after a period of two years' process monitoring are 89.4% and 91.1% respectively. CONCLUSIONS: The phantom measurements closely match the planned dose distributions, demonstrating that the calculation accuracy of the new Precision TPS and the delivery accuracy of the Radixact unit are adequate, with respect to international guidelines and reports. Furthermore, a first correlation with the planning parameters was made. Action and tolerance limits have been set for the new Radixact Linac.

Operating procedures, risk management and challenges during implementation of adaptive and non-adaptive MR-guided radiotherapy: 1-year single-center experience.

BACKGROUND: Main purpose was to describe procedures and identify challenges in the implementation process of adaptive and non-adaptive MR-guided radiotherapy (MRgRT), especially new risks in workflow due to the new technique. We herein report the single center experience for the implementation of (MRgRT) and present an overview on our treatment practice. METHODS: Descriptive statistics were used to summarize clinical and technical characteristics of treatment and patient characteristics including sites treated between April 2019 and end of March 2020 after ethical approval. A risk analysis was performed to identify risks of the online adaptive workflow. RESULTS: A summary of the processes on the MR-Linac including workflows, quality assurance and possible pitfalls is presented. 111 patients with 124 courses were treated during the first year of MR-guided radiotherapy. The most commonly treated site was the abdomen (42% of all treatment courses). 73% of the courses were daily online adapted and a high number of treatment courses (75%) were treated with stereotactic body irradiation. Only 4/382 fractions could not be treated due to a failing online adaptive quality assurance. In the risk analysis for errors, the two risks with the highest risk priority number were both in the contouring category, making it the most critical step in the workflow. CONCLUSION: Although challenging, establishment of MRgRT as a routinely used technique at our department was successful for all sites and daily o-ART was feasible from the first day on. However, ongoing research and reports will have to inform us on the optimal indications for MRgRT because careful patient selection is necessary as it continues to be a time-consuming treatment technique with restricted availability. After risk analysis, the most critical workflow category was the contouring process, which resembles the need of experienced staff and safety check paths.

Quantity and quality of complementary and alternative medicine recommendations in clinical practice guidelines for type 2 diabetes mellitus: A systematic review.

AIMS: Approximately 70% of Americans with diabetes have used complementary and alternative medicine (CAM) in the past year. Healthcare providers often receive minimal training on these therapies and subsequently rely on clinical practice guidelines (CPGs) to supplement their knowledge about the safe and effective use of CAM for the treatment/management of type 2 diabetes mellitus (T2DM). The purpose of this systematic review is to determine the quantity and assess the quality of CAM recommendations in CPGs for the treatment and/or management of T2DM. DATA SYNTHESIS: MEDLINE, EMBASE, and CINAHL were systematically searched from 2009 to 2020, in addition to the Guidelines International Network and the National Center for Complementary and Integrative Health websites. CPGs containing treatment and/or management recommendations for T2DM were eligible; those with CAM recommendations were quality-assessed with the AGREE II instrument twice, once for the overall CPG and once for the CAM sections. Twenty-seven CPGs were deemed eligible, of which 7 made CAM recommendations. Mean scaled domain percentages were (overall, CAM): scope and purpose (89.7%, 79.8%), clarity of presentation (85.7%, 48.4%), stakeholder involvement (67.9%, 28.2%), applicability (54.8%, 20.2%), rigour of development (49.7%, 35.7%), and editorial independence (44.1%, 44.1%). CONCLUSIONS: Quality varied within and across CPGs; domain scores across CAM sections generally scored lower than the overall CPG. Given that CAM therapies for T2DM are only represented in one-quarter of eligible CPGs and are of lower quality, a knowledge gap exists for healthcare providers who seek evidence-based information on this topic in order to effectively counsel inquiring patients.

Identification and Critical Appraisal of Food Service Satisfaction Questionnaires for Use in Nursing Homes: A Systematic Review.

BACKGROUND: Food service provision in nursing homes is a complex, adaptive system through which multiple stakeholders interface. Organizational stakeholders include staff involved in preparing and delivering meals. Consumer stakeholders are the end users including residents and family. Questionnaires can be an economical and efficient method of measuring food service satisfaction in nursing homes and a powerful quality improvement tool. OBJECTIVE: (1) To identify questionnaires that measure food service satisfaction of various stakeholders in a nursing homes and (2) to critically appraise the psychometric properties of identified questionnaires. METHODS: Five electronic databases were searched (Cumulative Index to Nursing and Allied Health Literature, Medline, ProQuest, Scopus, and Cochrane) in April 2020. Data from the eligible studies were extracted, and the psychometric properties were critically appraised using the Consensus-Based Standards for the Selection of Health Measurement Instruments. RESULTS: This review identified 129 studies that used a questionnaire to measure food service satisfaction in nursing homes. Of those, 107 studies representing 75 unique general nursing home satisfaction questionnaires were excluded for failing to adequately explore aspects related to food service. From the remaining 22 studies, 7 food service satisfaction questionnaires were identified; 5 intended for consumers (residents) and 2 intended for organizational stakeholders (staff). Using the Consensus-Based Standards for the Selection of Health Measurement Instruments quality criteria, most questionnaires had flaws in content validity and construct validity, primarily due to small sample sizes. No questionnaires explored food service satisfaction from the family perspective. CONCLUSIONS: Nursing homes collect satisfaction information for accreditation, marketing, benchmarking, and quality improvement. Although questionnaires are easy to administer, the quality of the data they collect is impacted by the validity and reliability of the questionnaires used. Using unreliable satisfaction data may mean that nursing homes are not accurately able to understand the impact of changes in the system on stakeholder satisfaction.

Modifying phrases in surgical pathology reports: introduction of Standardized Scheme of Reporting Certainty in Pathology Reports (SSRC-Path).

Pathologists often incorporate modifying phrases in their diagnosis to imply varying levels of diagnostic certainty; however, what is implied by the pathologists is not equivalent with what is perceived by the referring physicians and patients. This discordance can have significant implications in management, safety, and cost. We intend to identify lack of consistency in interpretation of modifying phrases by comparing perceived level of certainty by pathologists and non-pathologists, and introduce a standard scheme for reporting uncertainty in pathology reports using the experience with imaging reporting and data systems. In this study, a list of 18 most commonly used modifying phrases in pathology reports was distributed among separate cohorts of pathologists (N = 17) and non-pathology clinicians (N = 225) as a questionnaire survey, and the participants were asked to assign a certainty level to each phrase. All the participants had practice privileges in Brown University-affiliated teaching hospitals. The survey was completed by 207 participants (17 pathologists, 190 non-pathologists). It reveals a significant discordance between the interpretations of the modifying phrases between the two cohorts, with significant variations in subgroups of non-pathology clinicians. Also there is disagreement between pathologists and other clinicians regarding the causes of miscommunication triggered by pathology reports. Pathologists and non-pathology clinicians should be mindful of the potential sources of misunderstanding of pathology reports and take necessary actions to prevent and clarify the uncertainties. Using a standard scheme for reporting uncertainty in pathology reports is recommended.

Out-of-Hospital COVID-19 Deaths: Consequences for Quality of Medical Care and Accuracy of Cause of Death Coding.

Objectives. To examine age and temporal trends in the proportion of COVID-19 deaths occurring out of hospital or in the emergency department and the proportion of all noninjury deaths assigned ill-defined causes in 2020. Methods. We analyzed newly released (March 2021) provisional COVID-19 death tabulations for the entire United States. Results. Children (younger than 18 years) were most likely (30.5%) and elders aged 64 to 74 years were least likely (10.4%) to die out of hospital or in the emergency department. In parallel, among all noninjury deaths, younger people had the highest proportions coded to symptoms, signs, and ill-defined conditions, and percentage symptoms, signs, and ill-defined conditions increased from 2019 to 2020 in all age-race/ethnicity groups. The majority of young COVID-19 decedents were racial/ethnic minorities. Conclusions. The high proportions of all noninjury deaths among children, adolescents, and young adults that were coded to ill-defined causes in 2020 suggest that some COVID-19 deaths were missed because of systemic failures in timely access to medical care for vulnerable young people. Public Health Implications. Increasing both availability of and access to the best hospital care for young people severely ill with COVID-19 will save lives and improve case fatality rates.